Deep reinforcement learning (RL) can enable robots to autonomously acquire complex behaviors, such as legged locomotion. However, RL in the real world is complicated by constraints on efficiency, safety, and overall training stability, which limits its practical applicability. We present APRL, a policy regularization framework that modulates the robot's exploration over the course of training, striking a balance between flexible improvement potential and focused, efficient exploration. APRL enables a quadrupedal robot to efficiently learn to walk entirely in the real world within minutes and continue to improve with more training where prior work saturates in performance. We demonstrate that continued training with APRL results in a policy that is substantially more capable of navigating challenging situations and is able to adapt to changes in dynamics with continued training.

When children first learn to crawl, walk, and run it is a process full of trial and error -- expressed with frustrating cries and bumped heads. This tender learning process from early childhood may seem like an innately human experience, but it's actually incredibly similar to what engineers at the University of California, Berkeley sent their bipedal robot Cassie through in order to teach it to walk. Dancing and fighting robots, like those made by and parodied of Boston Dynamics' robots, have taken the internet by storm in the past few years. But what these videos don't show are the fine-tuned and choreographed movements often lurking in their code. Zhongyu Li is a Ph.D. candidate at the University of Berekely studying robotic locomotion.